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5 GEOMETRIC MODEL DATABASE FOR USE IN UBIQUITOUS COMPUTING 

BACKGROUND 

Technical Field: 

10 

The invention is related to a ubiquitous computing environment , and 
more particularly to a system and process for providing a geometric model 
database for use in aggregating information and responding to queries about the 
environment's geometric state 

15 

Background Art: 

Ubiquitous computing revolves around extending computational activities 
beyond the current desktop model and into the environment. In future homes 

2 0 and offices, access to computing should be as natural as access to lighting. 
Users should not be required to go to a special place (i.e., the desktop) to 
interact with the computer. Rather, the computer should be available to interface 
with the user anywhere in the home or office (or more generally anywhere in an 
arbitrarily large environment), through whatever set of devices is available, be 

2 5 they fixed or carried by the user. 

It is noted that the term computer is used loosely here in that the user 
actually would have access to a wide variety of computing and information 
services, which will likely employ many computers and "smart" devices such as 
30 PDA's, mobile phones, cameras, etc. For example, computing services such as 
web browsing, document editing, or video conferencing are envisioned. Thus, it 
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should be understood that when the term computer is used in connection with 
the concept of ubiquitous computing, in actuality many computers may be 
involved non-exclusively in a single interactive session. 

The usefulness of an ubiquitous computing system is greatly enhanced if 
it maintains an awareness of the users, including such things as their locations 
and orientation (i.e. which way they are facing). One goal of such a system 
would then be to understand the physical and functional relationship between 
the users and various I/O devices. Such knowledge could be employed to allow 
a user to move from room to room while still maintaining an interactive session 
with the computer. Through cameras, microphones and other perceptual 
devices or sensors, the user would always be in touch with the computer, no 
matter where the computer is physically located. Conversely, through displays, 
speakers, and other output devices, the computer will always be able to contact 
the user. For example, the system could track the location of a person and 
transfer a current computing session from a monitor in one room, to a different 
monitor in another room where the user enters, while the user continues to 
control the session via a PDA carried on his person. 

As can be imagined, a fundamental unifying aspect of ubiquitous 
computing is a representation of the physical relationships between people, 
things, and devices - in other words a geometric model of the environment. To 
date, no satisfactory geometric modeling program exists. 

The present invention provides this needed geometric modeling program 
in the form of a dynamic database that maintains information about the physical 
relationships between users, various I/O devices and sensors, as well as 
passive things in the world such as furniture, doors and walls. 
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It is noted that in the remainder of this specification, the description refers 
to various individual publications identified by a numeric designator contained 
within a pair of brackets. For example, such a reference may be identified by 
reciting, "reference [1]" or simply "[1]". Multiple references will be identified by a 
5 pair of brackets containing more than one designator, for example, [3, 18, 22, 9]. 
A listing of the publications corresponding to each designator can be found at 
the end of the Detailed Description section. 

10 SUMMARY 

The present invention is directed toward a system and process for 
providing a geometric model database for use in a ubiquitous computing 
environment. In general, the geometric model database system and process is 
1 5 capable of accepting information about the geometric state of the environment, 
building a geometric model of this environment, maintaining and storing the 
geometric model, and handling queries about the environment's geometric state. 

The task of building a geometric model begins by establishing a set of 
2 0 entities that are of interest in the environment. An entity represents an object 
which exists in the physical world (e.g., a person, I/O device, or sensor). In the 
geometric model, an entity is represented by a coordinate frame (or just "frame" 
for short) and an extent. The frame of an entity has a pre-selected origin 
somewhere on or within the object. Coordinate axes extend from this origin. In 
2 5 a preferred embodiment of the model, only the orthogonal x and y axes are used 
and represent the axes that could be "seen" looking down onto the environment. 
Entity identification information is provided to the geometric model program from 
external programs. When an entity is first identified, the geometric model 
program assigns an entity identifier (ID) that uniquely identifies the entity. This 
30 information is provided back to the outside program which introduced the entity, 
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and possibly to some other repository of identification information. The entity ID 
is used by outside programs when updating information about an entity or when 
making a query about the physical relationships between entities. Entities may 
be created or deleted and any time during the period which the model exists. 

5 

The location of an entity in the physical world is defined using 
"measurements". In general, a measurement is simply a mathematical 
description of the geometric relationship between two entities. More precisely, a 
measurement describes the position and orientation of one entity's coordinate 

10 frame, expressed in terms of another entity's coordinate frame. Measurements 
originating at an entity's frame are expressed in terms of that frame. In addition, 
measurements ending at an entity's frame, end at the origin of that frame. While 
various mathematical representations of the geometric relationship between 
entities could be employed, a preferred one characterizes a measurement as the 

15 relative position, and the relative orientation or heading, of two entities. 

A way to envision the foregoing method of representing the locations of 
the entities is to imagine that the frames of the entities are related by a chain of 
measurements. Sensors in the environment provide each link in the 

2 0 measurement chain. However, the sensor readings are processed outside the 
geometric model program and just the relative coordinates are provided as 
measurements to the model program. As with the entities, a measurement ID is 
assigned by the geometric model program to each new measurement provided 
to it. These measurement IDs are issued back to the program providing the 

2 5 measurement (and possibly to some outside database) so that any updates that 
are provided in the future can be identified as being associated with the previous 
measurement. Typically, a new measurement is used to replace the existing 
measurement with that ID. 
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Measurements in the geometric model preferably also reflect the spatial 
uncertainty associated with the method used to obtain them. One way of 
introducing an uncertainty estimate into the measurements is to employ a 
Gaussian uncertainty model where rather than a measurement being a particular 
5 value, it is provided to the geometric model program as a mean estimate for 
each coordinate and a covariance matrix for each that is indicative of the 
uncertainty of the coordinate mean. One advantage of characterizing the 
measurements in the foregoing way is that it can be used to enhance certain 
optional procedures performed within the geometric model program that are 

1 o designed to test the accuracy and revise the measurements provided to the 

program. One class of these methods employs the concept of cycle-based 
constraints. A cycle is a loop formed by connected entities frames. In the 
environment these loops represent measurements from a first entity frame 
(which could be any one in the loop) to the next and so on until a measurement 
15 connects the next to last entity frame back to the first entity frame. The 

significance of a cycle is that the measurements around the loop should return 
precisely to where they started, i.e. a "zero difference". If they do not then the 
previously provided uncertainty estimates could be used to adjust the 
measurements within the loop to obtain the desired zero difference. In this case, 

2 0 measurements are adjusted in proportion to the amount of uncertainty in each 

one. Specifically, the measurement cycles existing in the geometric model are 
identified. One of the entities making up each cycle, for which an actual location 
corresponding the origin of its frame is known, is selected as the first entity of 
the cycle. The difference between the actual location of the first entity frame's 
2 5 origin and the location of that origin as indicated by following the chain of 

measurements making up the cycle is then determined. Revised measurements 
for the identified cycles are computed next by simultaneously adjusting the 
measurements of all the identified cycles based on their associated uncertainty 
estimates, so as to make the difference between the actual location of the first 
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entity frame's origin in each of the identified cycles match the location of that 
origin as indicated by following the chain of measurements making up the cycle. 

It is further noted that there may be more than one measurement input 
from different sources to describe the geometric relationship between the same 
two entity frames. Each of the measurements would be assigned a different ID 
and so could co-exist in the same model. Thus there would be a question as to 
which measurement to use in responding to a query. One way to deal with this 
situation is to simply chose one of them arbitrarily for use in responding to a 
query, as was done in tested embodiment of the geometric model program. 
Another way would be to choose the measurement with the lowest uncertainty 
estimate. Alternatively, the two measurements could be combined using the 
relative uncertainties as weights in the combination process [1]. 

Another accuracy testing procedure that could be implemented based on 
the aforementioned cycles involves detecting inconsistencies. This would entail 
following the measurements forming a cycle around from a first entity frame to 
compute a location of the origin of the first frame, while using the uncertainty 
estimates associated with each measurement to compute an uncertainty region 
around the computed origin location. The first entity frame can be any one in the 
loop for which an actual location corresponding the origin of the frame has been 
previously specified, and the uncertainly region essentially defines a area 
around the computed location. If the specified location of the entity frame 
chosen as the first in the cycle falls within the uncertainty region, then the 
measurements are considered to be accurate. If, however, the specified location 
falls outside the aforementioned region, then it can be surmised that at least one 
of the measurements making up the cycle is incorrect. It would not be known 
which measurement or measurements in the cycle are incorrect. However, 
knowing that at least one of them is in error, the geometric model program could 
be designed to disregard all the measurements that are part of the cycle. 



Further, the geometric model program could provide feedback to the program or 
programs that inputted the suspect measurements and request that new 
measurements be provided. The inconsistency check would then be performed 
using the updated measurements. 

5 

The foregoing accuracy testing procedures can also be combined as 
follows. First, all the cycles of measurements existing in the geometric model 
are identified. Then, for each identified cycle, the previously-described process 
for detecting inconsistencies in the measurements of a cycle is performed. If it 

10 is determined that at least one of the measurements making up the cycle is 
inconsistent, then the measurements are ignored and potentially replaced. 
However, if the computed location of the first entity frame of the cycle is not 
exactly the same as the specified location but falls within the aforementioned 
uncertainty region, then revised measurements for the cycle are computed by 

15 simultaneously adjusting the measurements based on their associated 

uncertainty estimates so as to make the difference in the actual location of the 
first entity frame's origin in each of the identified cycles match the location of that 
origin as indicated by following the chain of measurements making up the cycle. 

2 0 It is also noted that any of the foregoing accuracy testing methods could 

be repeated each time a new or updated measurement is received. Or, 
alternately, the methods could be repeated on a periodic basis. 

It was mentioned previously that an entity is represented in the geometric 

2 5 model as a coordinate frame and an extent. Real world entities will have some 

physical size. In addition, a physical object could have some service region, 
such as a field of view associated with a camera or display device. It is 
advantageous to augment the geometric model with these region descriptions, 
which are called extents. Thus, an inputting program would include in its 

3 0 description of an entity any extents associated with the entity. The geometric 
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program would preferably treat each entity-extent combination as separate 
entities with a measurement of zero between them. Thus, each entity has only 
one extent, but multiple entities may refer to the same physical device. By 
default in the preferred embodiment, if no extent is explicitly specified for an 
5 entity, a point extent is assumed. This point extent can be defined as being 
located at the origin of the entity's frame, or at a location having some 
prescribed geometric relationship to the origin of the entity's frame. It is noted 
that the description of the area in the environment associated with an extent is 
expressed in terms of the coordinate frame of the corresponding entity. The 
1 0 extent of an entity can be expressed as having any planar geometric shape. 
However, in tested embodiments of the geometric model program the area 
associated with an extent was expressed as a polygon, (including degenerate 
cases such as a point when no extent is specified or a line segment such as in 
the case of a flat display screen which appears as a line from above). 

15 

The geometric model of the environment is constructed from three 
components input by outside programs. Namely, entities, extents associated 
with entities, and measurements between entity frames. However, the 
environment being modeled will typically change. Thus, it is necessary that the 

2 0 presence of an entity and the measurements between entity frames be updated 
on an on-going basis to keep the geometric model current. The aforementioned 
entity and measurement IDs are used in this updating process. In view of the 
need to keep the geometric model current an assumption is made that some 
mechanism is in place which causes the outside programs feeding the model 

2 5 program to input changes. As indicated previously, when an entity is created, an 
entity ID is assigned. Thus, any new entity introduced by an outside program 
would also be issued a unique entity ID. Whenever, a outside program refers to 
an entity it does so by using the assigned entity ID. As a consequence, the 
outside programs must have the capability to retain these IDs. If an outside 

30 program indicates that an entity is no longer in the environment, the entity ID is 
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deactivated and preferably not used again to prevent confusion. In a similar 
way, outside programs that input updated measurements would do so by 
referring to the appropriate measurement ID. The same measurement ID can be 
retained for the updated version as the ID refers to the existence of geometric 
5 relationship between two entities not a specific instance of that relationship. 
However, if an entity associated with a measurement is eliminated from the 
geometric model, all the measurements which refer to that entity should also be 
deleted, and preferably the measurement ID retired to prevent confusion. 

10 The geometric model database is preferably stored in a readily accessible 

memory, such as a RAM, while the geometric model program is running so as to 
facilitate updating the model and processing queries concerning the geographic 
relationships represented by the model. However, when the program is exited 
the model may be lost. Of course, the geometric model database could be 

15 rebuilt each time the program is executed. However, this requires the outside 
programs that feed entity, measurement and extent information into the 
geometric model program to supply this information each time. While a typical 
ubiquitous computing environment is dynamic in nature, many things do not 
change that often, such as the location of many I/O devices and sensors. Thus, 

2 0 there is some redundancy in re-building the model each time the geometric 
model program is executed that could be avoided. One way of avoiding this 
redundancy would be to store the relatively fixed or static portions of the model 
in a non-volatile memory and initialize the model using this information each time 
the geometric model program is executed. It is noted however, that since the 

2 5 entity IDs and measurement IDs associated with the static model data would be 
retained from one session to the next, any outside program that wishes to 
provide inputs or make queries would have to have the capability of permanently 
storing this information, or perhaps more practically, obtaining the IDs from a 
separate database maintained for that purpose. 
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In the event that some of the fixed portions of the geometric model 
change, or new fixed entities, etc., are added to the model, the initializing 
database could be updated accordingly so as to be as accurate as possible the 
next time the geometric model program is executed. 

5 

The primary purpose of creating and maintaining the geometric model of 
the ubiquitous computing environment is to respond to queries about the 
geometric relationships between entities. While a query could be as simple as 
what extent is associated with a particular entity, typically the requested 

10 information will be directed at the relationships between the various entities 
and/or their extents. There are two fundamental types of queries about these 
relationships that the geometric model program will be called upon to answer. 
The first type is a query concerning the relative relationship between two 
entities. There may be a single measurement already existing between the 

15 frames of the entities in question. If so, the answer to the query is 

straightforward. However, suppose there is no direct measurement between the 
between entities involved in the query. If at least one path of measurements 
exists between two entity frames, then a single geometric relationship can be 
derived between them. However, in such a case the geometric relationship must 

2 0 be derived from the multiple measurements making up the path. The first part of 
this process is to find a measurement path between the two entity frames in 
question. One way of doing this is to find a path having the fewest number of 
measurements between the query entities. This can be accomplished using a 
breadth-first search procedure. It is noted that if two paths have the same 

25 number of measurements (i.e., a tie), either could be chosen arbitrarily, or some 
selection criteria such as the collective uncertainty of the path could be 
employed to make the choice. The next phase of the response to the first type 
of query involving the geometric relationship between two entities is to compute 
a single measurement describing the relationship and output it to the querying 

30 program. 
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The second type of query concerns a request to find an entity or set of 
entities whose extents have a particular geometric relationship to a given entity. 
The first phase of this process is to define the relationship between the given 
5 entity frame and each of the other entity frames in terms of the given entity's 
coordinate frame. This is accomplished just as it was in responding to the first 
type of query, with the exception that it is repeated to find the relationship 
between the given entity frame and every other entity frame, not just for one 
other. The next phase in this type of query could generally be characterized as 

10 a region intersection problem. Essentially, some region associated with a given 
entity frame is specified. This region could be a newly defined region, or it might 
be the aforementioned extent of the entity. A newly defined region could be any 
desired planar shape, however, for convenience the following description will 
assume the newly defined region (or one corresponding to an entity's extent) is a 

15 polygon as was used in tested embodiments of the geometric model program. 

The extents associated with all the other entities would be transformed using the 
computed relationships into the given entity's coordinate frame. The region 
intersection technique would then be employed to determine if extents 
associated with the other entities intersect the defined region. Those entities 

2 0 whose extents are discovered to intersect the region would be reported to the 
program making the query. 

When the purpose for a query is to determine if one entity is in the field of 
view of another, such as where the querying program wants to display 

25 something intended for the person to see, the aforementioned rotational 
coordinates used to indicate the "heading" of an entity could be used in 
answering the query. For instance, if the rotation component associated with the 
person indicates that the display screen would be in his or her field of view, then 
this would be reported. If not, then it could be reported to the querying program 

30 that the person could not see the display screen. The same procedure could be 
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performed in the context of the display screen to ensure that it is oriented such 
that the person can see it. 

It is noted that a query could be of a one-time or "one-shot" nature, or 
may be a standing query. For a one-time query, the querying program would 
request the desired geometric information and receive a one-time response. 
However, a standing query would be one predicated on some event, and every 
time this event occurred, a response to the query would be provided. For 
example, the previously discussed query concerning whether a person could 
clearly see a display screen might be requested to be responded to every time 
the person of interest is added to the geometric model of the environment (e.g., 
enters a room). It is also noted that because the environment is dynamic, the 
response to such a query may have to be re-computed each time it is responded 
to, rather than relying on the previous response. 

In addition to the just described benefits, other advantages of the present 
invention will become apparent from the detailed description which follows 
hereinafter when taken in conjunction with the drawing figures which accompany 
it. 

DESCRIPTION OF THE DRAWINGS 

The specific features, aspects, and advantages of the present invention 
will become better understood with regard to the following description, appended 
claims, and accompanying drawings where: 

FIG. 1 is a diagram depicting a general purpose computing device 
constituting an exemplary system for implementing the present invention. 
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FIG. 2 is a flow chart diagramming an overall process for providing a 
geometric model database for use in a ubiquitous computing environment to 
respond to queries about the environment's geometric state in accordance with 
the present invention. 

5 

FIG. 3 is a flow chart diagramming a process for building the geometric 
model database, in accordance with the model building module of the overall 
process of Fig. 2. 

10 FIG. 4 is a diagram illustrating a preferred measurement scheme 

employed in the present invention to describe the location of an entity in the 
environment, where each measurement describes the position and orientation of 
one entity's coordinate frame, expressed in terms of another entity's coordinate 
frame. 

15 

FIG. 5 is a diagram illustrating an example of a chain of measurements 
linking a series of entities. 



FIG. 6 is a flow chart diagramming a first error detection process for 
2 0 identifying and revising measurements that can be used in conjunction with the 
model building module of the overall process of Fig. 2. 

FIG. 7 is a flow chart diagramming a second error detection process for 
identifying and handling errors in measurements that can be used in conjunction 
2 5 with the model building module of the overall process of Fig. 2. 

FIG. 8 is a flow chart diagramming a process for responding to queries 
concerning the geometric relationship between two entities modeled in the 
geometric model database, in accordance with the response module of the 
30 overall process of Fig. 2. 
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FIG. 9 is a flow chart diagramming process for responding to queries 
concerning a request to find an entity or entities whose extents have a particular 
geometric relationship to a prescribed region or extent associated with a given 
5 entity, in accordance with the response module of the overall process of Fig. 2. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 



10 In the following description of the preferred embodiments of the present 

invention, reference is made to the accompanying drawings which form a part 
hereof, and in which is shown by way of illustration specific embodiments in 
which the invention may be practiced. It is understood that other embodiments 
may be utilized and structural changes may be made without departing from the 

1 5 scope of the present invention. 



Before providing a description of the preferred embodiments of the 
present invention, a brief, general description of a suitable computing 
environment in which the invention may be implemented will be described. 

2 0 Figure 1 illustrates an example of a suitable computing system environment 100. 
The computing system environment 100 is only one example of a suitable 
computing environment and is not intended to suggest any limitation as to the 
scope of use or functionality of the invention. Neither should the computing 
environment 100 be interpreted as having any dependency or requirement 

25 relating to any one or combination of components illustrated in the exemplary 
operating environment 100. 



The invention is operational with numerous other general purpose or 
special purpose computing system environments or configurations. Examples of 
30 well known computing systems, environments, and/or configurations that may be 
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suitable for use with the invention include, but are not limited to, personal 
computers, server computers, hand-held or laptop devices, multiprocessor 
systems, microprocessor-based systems, set top boxes, programmable 
consumer electronics, network PCs, minicomputers, mainframe computers, 
5 distributed computing environments that include any of the above systems or 
devices, and the like. 

The invention may be described in the general context of computer- 
executable instructions, such as program modules, being executed by a 

10 computer. Generally, program modules include routines, programs, objects, 
components, data structures, etc. that perform particular tasks or implement 
particular abstract data types. The invention may also be practiced in distributed 
computing environments where tasks are performed by remote processing 
devices that are linked through a communications network. In a distributed 

15 computing environment, program modules may be located in both local and 
remote computer storage media including memory storage devices. 

With reference to Figure 1 , an exemplary system for implementing the 
invention includes a general purpose computing device in the form of a computer 

2 0 110. Components of computer 110 may include, but are not limited to, a 

processing unit 120, a system memory 130, and a system bus 121 that couples 
various system components including the system memory to the processing unit 
120. The system bus 121 may be any of several types of bus structures 
including a memory bus or memory controller, a peripheral bus, and a local bus 

25 using any of a variety of bus architectures. By way of example, and not 

limitation, such architectures include Industry Standard Architecture (ISA) bus, 
Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video 
Electronics Standards Association (VESA) local bus, and Peripheral Component 
Interconnect (PCI) bus also known as Mezzanine bus. 
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Computer 1 10 typically includes a variety of computer readable media. 
Computer readable media can be any available media that can be accessed by 
computer 110 and includes both volatile and nonvolatile media, removable and 
non-removable media. By way of example, and not limitation, computer 
5 readable media may comprise computer storage media and communication 
media. Computer storage media includes both volatile and nonvolatile, 
removable and non-removable media implemented in any method or technology 
for storage of information such as computer readable instructions, data 
structures, program modules or other data. Computer storage media includes, 

10 but is not limited to, RAM, ROM, EEPROM, flash memory or other memory 

technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, 
magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic 
storage devices, or any other medium which can be used to store the desired 
information and which can accessed by computer 110. Communication media 

15 typically embodies computer readable instructions, data structures, program 

modules or other data in a modulated data signal such as a carrier wave or other 
transport mechanism and includes any information delivery media. The term 
"modulated data signal" means a signal that has one or more of its 
characteristics set or changed in such a manner as to encode information in the 

2 0 signal. Byway of example, and not limitation, communication media includes 

wired media such as a wired network or direct-wired connection, and wireless 
media such as acoustic, RF, infrared and other wireless media. Combinations of 
the any of the above should also be included within the scope of computer 
readable media. 

25 

The system memory 130 includes computer storage media in the form of 
volatile and/or nonvolatile memory such as read only memory (ROM) 131 and 
random access memory (RAM) 132. A basic input/output system 133 (BIOS), 
containing the basic routines that help to transfer information between elements 

3 0 within computer 110, such as during start-up, is typically stored in ROM 131 . 
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RAM 132 typically contains data and/or program modules that are immediately 
accessible to and/or presently being operated on by processing unit 120. By 
way of example, and not limitation, Figure 1 illustrates operating system 134, 
application programs 135, other program modules 136, and program data 137. 

The computer 110 may also include other removable/non-removable, 
volatile/nonvolatile computer storage media. By way of example only, Figure 1 
illustrates a hard disk drive 141 that reads from or writes to non-removable, 
nonvolatile magnetic media, a magnetic disk drive 151 that reads from or writes 
to a removable, nonvolatile magnetic disk 152, and an optical disk drive 155 that 
reads from or writes to a removable, nonvolatile optical disk 156 such as a CD 
ROM or other optical media. Other removable/non-removable, 
volatile/nonvolatile computer storage media that can be used in the exemplary 
operating environment include, but are not limited to, magnetic tape cassettes, 
flash memory cards, digital versatile disks, digital video tape, solid state RAM, 
solid state ROM, and the like. The hard disk drive 141 is typically connected to 
the system bus 121 through an non-removable memory interface such as 
interface 140, and magnetic disk drive 151 and optical disk drive 155 are 
typically connected to the system bus 121 by a removable memory interface, 
such as interface 150. 

The drives and their associated computer storage media discussed above 
and illustrated in Figure 1, provide storage of computer readable instructions, 
data structures, program modules and other data for the computer 110. In 
Figure 1, for example, hard disk drive 141 is illustrated as storing operating 
system 144, application programs 145, other program modules 146, and 
program data 147. Note that these components can either be the same as or 
different from operating system 134, application programs 135, other program 
modules 136, and program data 137. Operating system 144, application 
programs 145, other program modules 146, and program data 147 are given 



17 



different numbers here to illustrate that, at a minimum, they are different copies. 
A user may enter commands and information into the computer 1 10 through 
input devices such as a keyboard 162 and pointing device 161, commonly 
referred to as a mouse, trackball or touch pad. Other input devices (not shown) 
5 may include a microphone, joystick, game pad, satellite dish, scanner, or the 
like. These and other input devices are often connected to the processing unit 
120 through a user input interface 160 that is coupled to the system bus 121 , but 
may be connected by other interface and bus structures, such as a parallel port, 
game port or a universal serial bus (USB). A monitor 191 or other type of 

10 display device is also connected to the system bus 121 via an interface, such as 
a video interface 190. In addition to the monitor, computers may also include 
other peripheral output devices such as speakers 197 and printer 196, which 
may be connected through an output peripheral interface 195. Of particular 
significance to the present invention, a camera 163 (such as a digital/electronic 

15 still or video camera, or film/photographic scanner) capable of capturing a 

sequence of images 164 can also be included as an input device to the personal 
computer 1 1 0. Further, while just one camera is depicted, multiple cameras 
could be included as input devices to the personal computer 110. The images 
164 from the one or more cameras are input into the computer 1 10 via an 

2 0 appropriate camera interface 165. This interface 165 is connected to the system 
bus 121, thereby allowing the images to be routed to and stored in the RAM 132, 
or one of the other data storage devices associated with the computer 110. 
However, it is noted that image data can be input into the computer 1 10 from any 
of the aforementioned computer-readable media as well, without requiring the 

2 5 use of the camera 1 63. 



The computer 110 may operate in a networked environment using logical 
connections to one or more remote computers, such as a remote computer 180. 
The remote computer 180 may be a personal computer, a server, a router, a 
3 0 network PC, a peer device or other common network node, and typically 
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includes many or all of the elements described above relative to the computer 
110, although only a memory storage device 181 has been illustrated in Figure 
1. The logical connections depicted in Figure 1 include a local area network 
(LAN) 171 and a wide area network (WAN) 173, but may also include other 
5 networks. Such networking environments are commonplace in offices, 
enterprise-wide computer networks, intranets and the Internet. 

When used in a LAN networking environment, the computer 1 10 is 
connected to the LAN 171 through a network interface or adapter 170. When 

10 used in a WAN networking environment, the computer 1 10 typically includes a 
modem 172 or other means for establishing communications over the WAN 173, 
such as the Internet. The modem 172, which may be internal or external, may 
be connected to the system bus 121 via the user input interface 160, or other 
appropriate mechanism. In a networked environment, program modules 

15 depicted relative to the computer 1 10, or portions thereof, may be stored in the 
remote memory storage device. By way of example, and not limitation, Figure 1 
illustrates remote application programs 185 as residing on memory device 181. 
It will be appreciated that the network connections shown are exemplary and 
other means of establishing a communications link between the computers may 

20 be used. 

The exemplary operating environment having now been discussed, the 
remaining part of this description will be devoted to a discussion of the program 
modules embodying the aforementioned system and process for providing a 

25 geometric model database for use in an ubiquitous computing environment. In 
general, the geometric model program is accomplished via the following process 
actions, as shown in the high-level flow diagram of Fig. 2. First, in process 
action 200, information about the geometric state of the environment is obtained. 
As will be described in more detail later, this action preferably involves outside 

3 0 programs providing identifying information concerning objects or persons 
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(referred to hereinafter as entities), which are to be included in the geometric 
model database, in addition, the outside programs would specify the extent of 
the identified entity and provide measurement information associated with the 
location of the entity. Both the extent and measurement concepts will be fully 
5 described in the sections that follow. A geometric model database of the 

environment is built based on the information obtained (process action 202), and 
then queries concerning the geometric relationships between entities in the 
environment are responded to using the geometric model database (process 
action 206). In addition, the geometric model database can be maintained by 
10 periodically modifying it based on the input of updated information about the 

geometric state of the environment (process action 204). This updated database 
is then used to respond to the aforementioned queries. Each of these tasks will 
be described in detail in the sections that follow. 

15 1.0 Building a Geometric Model of The Environment 

Referring to Fig. 3, the task of building a geometric model begins by 
establishing a set of entities that are of interest in the environment (process 
action 300). An entity represents an object which exists in the physical world 

20 (e.g., a person, I/O device, or sensor). In addition, a single physical object may 
have different components (e.g., a laptop computer with both screen and 
keyboard sections) that could be included in the geometric model as separate 
entities. An entity is represented in the geometric model by a coordinate frame 
(or just "frame" for short) having a pre-selected origin contained on or within the 

25 entity, and an extent (process action 302). In a preferred embodiment of the 
model, only the orthogonal x and y axes are used and represent the axes that 
could be "seen" looking down onto the environment. For example, the origin of 
the frame of a display screen might be define as being in the middle of the 
screen as viewed from above, with one axis parallel to the screen and one axis 

30 perpendicular to the screen. Another example would be a person. A person 
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could be represented in the model by a frame origin coinciding with their center 
of mass and having one axis pointing in a positive direction to the person's right 
and the other axis pointing in a positive direction straight ahead of the person. 
Entity identification information is provided to the geometric model program from 
5 "outside" programs. When an entity is first identified, the geometric model 

program assigns an entity ID that uniquely identifies the entity. This information 
is provided back to the outside program which introduced the entity, and 
possibly to some other repository of identification information. The entity ID is 
used by outside programs when updating information about an entity or when 

1 o making a query about the physical relationships between entities. 

The location of an entity in the physical world is defined using 
"measurements". In general, a measurement is simply a mathematical 
description of the geometric relationship between two entities. While various 
15 mathematical representations of the geometric relationship between entities 

could be employed, a preferred one characterizes a measurement as the relative 
position, and the relative orientation or heading, of two entities. More precisely, 
a measurement describes the position and orientation of one entity's coordinate 
frame, expressed in terms of another entity's coordinate frame. Measurements 

2 0 originating at an entity's frame are expressed in terms of that frame. In addition, 

measurements ending at an entity's frame, end at the origin of that frame. Any 
program providing a measurement relating two entities is assumed to know the 
origin of the coordinate frame of the entities. For example, any program 
providing measurements which relate the frame of the display screen described 

25 above to another frame must be sure that the sensor being used does, in fact, 
make it's measurements relative to center of the screen, or the program must 
internally transform the measurement so that this is true. Thus, referring to Fig. 
3 once again, the next process action 304 performed in building a geometric 
model is to assign at least one measurement defining the entity's geometric 

30 relationship with one or more other entities. This effectively describes it's 
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location. Multiple measurements associated with a single entity would occur 
where separate measurements are supplied between the entity and more than 
one other entity. 

5 An example of the foregoing measurement scheme is illustrated in Fig. 4, 

The location of an entity (hereinafter referred to as the destination entity as the 
measurement ends at the entity), whose frame 402 is shown in the upper right 
hand corner, in relation to an originating entity (so called as the measurement 
originates from that entity), whose frame 400 is shown in the lower left hand 

10 corner, could be defined by a set of x,y coordinates 404, 406 as viewed from the 
coordinate frame 400 associated with the originating entity. Specifically, as 
shown in Fig. 4, the measurement between the frames 400 and 402 of the two 
entities would include a Ax coordinate 408 associated with the distance between 
the origins of the two frames along the x-axis 404 of the originating entity's frame 

15 400, and a Ay coordinate 410 associated with the distance between the origins 
of the two frames along the y-axis 406 of the originating entity's frame 400. The 
measurement between the frames 400 and 402 of the two entities would also 
include a A0 coordinate 412 associated with the degree of rotation of the 
destination entity's frame 402 in relation to the originating entity's frame 400. 

2 o This rotational component is a measure of the relative rotation of the destination 
entity's frame 402 relative to the originating entity's frame 400 about the axis 
defined by the positive cross product of the x and y axes 404, 406, assuming a 
right-handed coordinate system. The rotational component can be quite useful 
for entities such as a display screen or a person as it can be used to determine 

25 which way the entity is facing. For example, if a display screen where directed 
away from where a person is facing, the person would not be able to see the 
screen. Thus just knowing the relative positions of the screen and the person 
would not be enough to determine if the person could see the screen. 
Furthermore, there is no a priori reason to expect that all entities have 
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coordinate frames with their X and Y axes aligned. Thus, a rotational component 
is needed. 

A way to envision the foregoing method of representing the locations of 
5 the entities is to imagine that the frames of the entities are related by a chain of 
measurements. Sensors in the environment provide each link in the 
measurement graph. However, the raw sensor readings are processed outside 
the geometric model program and just the relative coordinates are provided as 
measurements to the model program (along with uncertainty estimates as 

10 discussed below). As with the entities, a measurement ID is assigned by the 
geometric model program to each new measurement provided to it. These 
measurement IDs are issued back to the program providing the measurement 
(and possibly to some outside database) so that any updates that are provided 
in the future can be identified as being associated with the previous 

15 measurement. Typically, a new measurement is used to replace the existing 
measurement with that ID. The foregoing chain of measurements model is 
illustrated in graph of Fig. 5. The lettered vertices 500 in Fig. 5 respectively 
represent the origin of each entity frame, and the lines 502 connecting the 
vertices represent the existence of a measurement of the geometric relationship 

2 0 between the connected entities' frames. It is noted that the lines 502 do not 

imply that the measurements were taken from (i.e., observed by one of) either of 
the physical embodiments of either of the pair of connected entities. Rather, 
they simply represent a relative measurement between the frames of two 
entities, and may have been derived from a sensor at another location in the 

2 5 environment. It is also noted that the measurements are invertible between two 
entity frames in the graph. Thus, a measurement in the graph can represent not 
only the relative position between entity As frame and entity B's frame, but also 
between entity B's frame and entity As frame. 
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it is noted that there is a possibility that more than one measurement 
would be input from different sources to describe the geometric relationship 
between the same two entity frames. Each of the measurements would be 
assigned a different ID and so could co-exist in the same model. Thus there 
5 would be a question as to which measurement to use in responding to a query. 
One way to deal with this situation is to simply chose one of them arbitrarily for 
use in responding to a query, as was done in tested embodiment of the 
geometric model program. Alternately, as will be discussed next, the 
measurement having the lowest uncertainty estimate could be chosen and the 
10 others ignored. Alternatively, the two measurements could be combined using 
the relative uncertainties as weights in the combination process [1]. 

Measurements in the geometric model preferably reflect the spatial 
uncertainty associated with the method used to obtain them. For example, a 

1 5 measurement taken directly by a user might be considered more certain that the 
same measurement derived from images of the scene. One way of introducing 
an uncertainty estimate into the measurements is to employ a Gaussian 
uncertainty model where rather than a measurement being a particular value, it 
is provided to the geometric model program as a mean estimate for each 

20 coordinate and a covariance matrix for each that is indicative of the uncertainty 
of the coordinate mean [1]. The advantage of characterizing the measurements 
in the foregoing way is that it can be used to enhance certain optional 
procedures performed within the geometric model program that are designed to 
test the accuracy and possibly even revise the measurements provided to the 

2 5 program. 

One class of error detection methods employs the concept of cycle-based 
constraints. Cycles can be best envisioned in reference Fig. 5. A cycle is a loop 
formed by connected entities frames. In Fig. 5, there are many such loops 
30 including CDEl, FGJK, FGK, GJK, BCDEFKH, and BCIEFKH, among others. In 
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the environment these loops represent measurements from the origin of a first 
entity frame (which could be any one in the loop) to the next and so on until a 
measurement connects the next to last entity frame back to the origin of the first 
entity frame. The significance of a cycle is that the measurements around the 
5 loop should return precisely to where they started, i.e. a "zero difference". If 

they do not then the previously provided uncertainty estimates could be used to 
adjust the measurements within the loop to obtain the desired zero difference. A 
conventional method for accomplishing this adjustment procedure is described in 
reference [2]. It is noted that this adjustment process could be repeated each 

10 time a new, updated measurement associated with a loop is received. Or 

alternately, the adjustment process could be repeated on a periodic basis to test 
the accuracy and revise the measurements in light of any new measurements 
that have been received. Further, it is noted that the adjustment procedure of 
reference [2] would produce only small changes, if any, for measurements with a 

1 5 high degree of certainty (such as those that where physically measured), 

whereas larger changes could be expected for those measurements having a 
low certainty. 

Thus, referring to Fig. 6, the foregoing version of an error detection 

2 0 process, which could optionally be employed with the present geometric 

modeling program, would generally involve first identifying cycles of 
measurements within the measurement data (process action 600). Next, for 
each of the identified cycles, a difference is computed between a given location 
of an arbitrarily chosen first entity , as originally provided to the geometric 
25 modeling database, and the location of the chosen first entity's frame, as 

indicated by following the chain of measurements around the cycle (process 
action 602). Revised versions of one or more of the measurements making up 
the identified cycles are computed by simultaneously adjusting the 
measurements based on the previously-described uncertainty estimates 

3 0 associated with the measurements so as to make the location of the chosen first 
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entity of each of the identified cycles as indicated by following the chain of 
measurements around the cycles to match the given location of those entities 
(process action 604). The reason for adjusting all the measurements of the 
identified cycles simultaneously is that a particular measurement may be part of 
5 more than one cycle. Simultaneously adjusting all the measurements ensures a 
"zero difference" is created for all the cycles, even those with shared 
measurements. 

Another error detection procedure that could be implemented based on 
10 the aforementioned cycles involves detecting inconsistencies in the 

measurements. Referring to Fig. 7, the first process action 700 of this alternate 
error detection procedure entails identifying cycles of measurements within the 
measurement data. Next, one of the identified cycles is selected (process action 
702), and choosing one of the cycle's entity frames as the first frame (process 
15 action 704). The first entity frame can be any one in the loop for which an actual 
location corresponding the origin of the frame has been previously specified. 
The measurements forming a selected cycle are then followed around the cycle 
from the chosen first entity frame to compute the location of the origin of the 
selected entity frame, while using the uncertainty estimates associated with each 

2 0 measurement to define an uncertainty region around the computed origin 

location (process action 706). The uncertainly region essentially defines an 
area around the computed location. It is then determined if the specified 
location of the entity frame chosen as the first in the cycle falls within the 
computed uncertainty region, as indicated by process action 708. If so, then the 
25 measurements are considered to be accurate (process action 710). If, however, 
the specified location falls outside the uncertainty region, then it is deemed that 
at least one of the measurements making up the cycle is incorrect (process 
action 712). It would not be known which measurement or measurements in the 
cycle are incorrect. However, knowing that at least one of them is in error, the 

3 0 geometric model program could be designed to disregard all the measurements 
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that are part of the cycle (process action 714). Further, the geometric model 
program could provide feedback to the program or programs that inputted the 
suspect measurements and request that new measurements be provided 
(process action 716). In either case, the error detection procedure is repeated 
5 for each remaining cycle (process action 718). 

It is noted that the error detection procedures outlined in Figs. 6 and 7 
could be combined as well. In this combined embodiment, the procedure would 
follow the actions outlined in Fig. 6. If it is determined that at least one of the 

10 measurements making up a cycle under consideration is inconsistent, then the 
measurements are ignored and potentially replaced. However, if the computed 
location of the first entity frame of the cycle is not exactly the same as the 
specified location but falls within the aforementioned uncertainty region, then 
revised versions of the measurements making up the identified cycles would be 

15 computed. 

Finally, the foregoing error detection procedures (Fig. 6 or Fig. 7 or the 
combined version) could be performed anytime a new measurement is provided 
to the geometric model database. Thus, for example, in the case of the 
2 0 procedure outlined in Fig. 7 where new measurements are requested when an 
error is found, the procedure would be performed again when the new 
measurements are received. Or, as mentioned previously, these procedures 
could be repeated on a periodic basis. 

25 It was mentioned previously that an entity is represented in the geometric 

model as a coordinate frame and an extent. The extent of an entity will now be 
described. Real world entities will have some physical size. In addition, an 
physical object could have some service region, such as a field of view 
associated with a camera or display device. It is advantageous to augment the 

30 geometric model with these region descriptions, which are called extents. Thus, 
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an inputting program would include in its description of an entity any extents 
associated with the entity. The geometric program would preferably treat each 
entity-extent combination which is related to the same physical item as separate 
entities with a measurement of zero between them. Thus, each entity has only 
5 one extent, but multiple entities may represent the same physical device in the 
model. By default in the preferred embodiment, if no extent is explicitly specified 
for an entity, a point extent is assumed. This point extent can be defined as 
being located at the origin of the entity's frame, or at a location having some 
prescribed geometric relationship to the origin of the entity's frame. It is noted 

1 0 that the description of the area in the environment associated with an extent is 
expressed in terms of the coordinate frame of the corresponding entity. The 
significance of this entity-extent approach will become apparent in the upcoming 
discussion of querying the geometric model program about geometric 
relationship in the environment. The extent of an entity can be expressed as 

15 having any planar geometric shape. However, in tested embodiments of the 

geometric model program the area associated with an extent was expressed as 
a polygon, (including degenerate cases such as a point when no extent is 
specified or a line segment such as in the case of a flat display screen which 
appears as a line from above) 

20 

2.0 Maintaining the Geometric Model 

As described previously, the geometric model of the environment is 
constructed from three components input by outside programs. Namely, entities, 

2 5 extents associated with entities and measurements between entity frames. 

However, the environment being modeled will typically change. For example, a 
new person could enter a room in the environment or someone could leave. 
Similarly, a new piece of I/O equipment could be added, or an old one replaced. 
Thus, entities will come and go, and the geometric model must be updated as to 

30 the current entities existing in the environment, as well as to their current 
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extents. In addition, entities can be expected to move about in the environment. 
For example, a person will typically move about a room. In addition, equipment 
can be relocated. Thus, it is necessary that the measurements between entity 
frames also be updated on an on-going basis to keep the geometric model 
5 current. The aforementioned entity and measurement IDs are used in this 

updating process. In view of the need to keep the geometric model current an 
assumption is made that some mechanism is in place which causes the outside 
programs feeding the model program to input changes. As indicated previously, 
when an entity is created, an entity ID is assigned. Thus, any new entity 

1 o introduced by an outside program would also be issued a unique entity ID. 

Whenever, a outside program refers to an entity it does so by using the 
assigned entity ID. As a consequence, the outside programs must have the 
capability to retain these IDs. If an outside program indicates that an entity is no 
longer in the environment, the entity ID is deactivated and preferably not used 
15 again to prevent confusion. In a similar way, outside programs that input 
updated measurements would do so by referring to the appropriate 
measurement ID. The same measurement ID can be retained for the updated 
version as the ID refers to the existence of geometric relationship between two 
entities not a specific instance of that relationship. However, if an entity 

2 0 associated with a measurement is eliminated from the geometric model, all the 

measurements corresponding to that entity should also be deleted, and 
preferably the measurement ID retired to prevent confusion. 

3.0 Storing the Geometric Model 

25 

The geometric model database is preferably stored in a readily accessible 
memory, such as a RAM, while the geometric model program is running so as to 
facilitate updating the model and processing queries concerning the geographic 
relationships represented by the model. However, when the program is exited 

3 0 the model may be lost. Of course, the geometric model database could be 
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rebuilt each time the program is executed. However, this requires the outside 
programs that feed entity, measurement and extent information into the 
geometric model program to supply this information each time. While a typical 
ubiquitous computing environment is dynamic in nature, many things do not 
5 change that often, such as the location of many I/O devices and sensors. Thus, 
there is some redundancy in re-building the model each time the geometric 
model program is executed that could be avoided. One way of avoiding this 
redundancy would be to store the relatively fixed or static portions of the model 
in a non-volatile memory and initialize the model using this information each time 

10 the geometric model program is executed. It is noted however, that since the 
entity IDs and measurement IDs associated with the static model data would be 
retained from one session to the next } any outside program that wishes to 
provide inputs or make queries would have to have the capability of permanently 
storing this information, or perhaps more practically, obtaining the IDs from a 

1 5 separate database maintained for that purpose. 

In the event that some of the fixed portions of the geometric model 
change, or new fixed entities, etc., are added to the model, the initializing 
database could be updated accordingly so as to be as complete as possible the 
2 0 next time the geometric model program is executed. 

4.0 Handling Queries Concerning the Relationships Between Entities . 

The primary purpose of creating and maintaining the geometric model of 
2 5 the ubiquitous computing environment is to respond to queries about the 

geometric relationships between entities. While a query could be as simple as 
what extent is associated with a particular entity, typically the requested 
information will be directed at the relationships between the various entities 
and/or their extents. 

30 
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There are two fundamental types of queries about these relationships that 
the geometric model program will be called upon to answer. The first type is a 
query concerning the relative relationship between two entities. For example, 
the query may be a request to specify where entity B (e.g., a display screen) is in 
5 relation to entity A (e.g., a person). There may be a single measurement already 
existing between the frames of the entities in question. If so the answer to the 
query is straight forward. However, suppose there is no direct measurement 
between the between entities involved in the query. If at least one path of 
measurements exists between two entity frames, then a single geometric 

10 relationship can be derived between them. However, in such a case the 

geometric relationship must be derived from the multiple measurements making 
up the path. The first part of this process is to find the measurement path 
between the two entity frames in question. One way of doing this is to find a 
path having the fewest number of measurements between the query entities. 

15 This can be accomplished using a breadth-first search procedure. Essentially, 
a breadth-first search entails starting at the primary entity frame in the query 
(such as the frame associated with entity A in the example above) and 
identifying all the entity frames that have measurements between themselves 
and primary entity frame. The process continues by choosing each of the 

20 identified entity frames in turn and then finding and keeping track of all the entity 
frames having measurements between themselves and the identified entity 
frame. This stepwise process continues until one of the identified entity frames 
is associated with the other entity specified in the query. The path of 
measurements back to the primary entity frame is then identified. For example, 

2 5 referring to Fig. 5, suppose a query asked where entity L was in relation to entity 
J. First the entity frames connected to J would be identified, i.e., entity frame K 
and entity frame G in this case. Next the entity frames connected to the newly 
identified entity frames would be found. In this case, entity frame K is connected 
to entity frames F 5 G and H, while entity frame G is connected to entity frames F, 

30 K and L. Since entity frame L has been identified the search is over and the 
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measurement path JGL is identified. This should be the path having the 
smallest number of measurements. It is noted that if two paths have the same 
number of measurements (i.e., a tie), either could be chosen arbitrarily, or some 
selection criteria such as the collective uncertainty of the path could be 
5 employed to make the choice. The next phase of the response to the first type 
of query involving the geometric relationship between two entities is to compute 
a single measurement describing the relationship and output it to the querying 
program. As mentioned previously, if there is a single measurement existing 
between the frames of the query entities, it can be used directly to respond to 
10 the query. However, when a path having multiple measurements is identified 
between the frames of the query entities, the measurements must be combined 
or "fused" to yield a response. Any conventional method of accomplish this 
fusion can be employed, such as for example the process described in reference 

Mi- 
ls 

The foregoing process for handling queries concerning the relative 
geometric relationship between two entities of the database is summarized in the 
flow diagram of Fig. 8. The first action 800 of the process involves waiting for 
queries concerning the relative geometric relationship between two entities. 

20 Periodically, it is determined whether such a query was received as indicated in 
process action 802. If no query was received, the waiting and checking actions 
800 and 802 are continued. If, however, such a query was received, then it is 
ascertained whether or not a direct measurement exist in the database between 
the two entities that are the subject of the query (process action 804). If so, then 

25 the requested measurement is provided to the source of the query (process 

action 806). Whenever there is no direct measurement available, a breadth-first 
search is employed to find a measurement path between the two entities 
involved in the query that has the fewest number of measurement links (process 
action 808). It is next determined in process action 810 whether a path was 

30 found at all. If no path was found, then the source of the query is informed that 
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the requested information cannot be provided (process action 812). However, if 
a path was found, then the requested measurement is computed using the 
measurements in the discovered path (process action 814), and the requested 
measurement is provided to the source of the query (process action 816). The 
5 monitoring is continued and foregoing procedure repeated every time a new 
query concerning the relative geometric relationship between two entities is 
received. 

The second type of query concerns a request to find an entity or set of 
10 entities whose extents have a particular geometric relationship to a given entity. 
The first phase of this process is to define the relationship between the given 
entity frame and each of the other entity frames in terms of the given entity's 
coordinate frame. This is accomplished just as it was in responding to the first 
type of query, with the exception that it is repeated to find the relationship 
1 5 between the given entity frame and every other entity frame, not just for one 
other. There may be cases where the relative geometric relationship between 
the frame of the given entity and the frame of one of the other entities involved in 
the query could not be determined due to a lack of the measurement path 
needed to compute the relationship. This circumstance is noted and reported to 
2 0 the query source in the manner described below. 

The next phase in this type of query could generally be characterized as a 
region intersection problem. Essentially, some region associated with a given 
entity frame is specified. This region could be a newly defined region, or it might 

25 be the aforementioned extent of the entity associated with the frame. A newly 

defined region could be any desired planar shape, however, for convenience the 
following description will assume the newly defined region (or one corresponding 
to an entity's extent) is a polygon as was used in tested embodiments of the 
geometric model program. As an example of where a newly defined region 

30 comes into play, suppose a query asks for all the entities which have extents 
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within some prescribed distance of a given entity. The geometric relationships 
between the given entity and all others in the environment are computed, as 
described above. Then, a polygonal region surrounding the given entity would 
be defined based on the prescribed distance and in terms of the given entity's 
5 coordinate frame. The extents associated with all the other entities would also 
be transformed using the computed relationships into the given entity's 
coordinate frame. A standard polygon intersection technique (or similar 
procedure if the region is characterized by a planar shape other than a polygon) 
would then be employed to determine if extents associated with the other entities 
l o intersect the defined region. Those entities whose extents are discovered to 
intersect the region would be reported to the program making the query. 

An example of a query where the extent of a given entity is involved would 
be as follows. Suppose a query asked what entities are at least partially in the 

15 field of view of a camera, where the camera is an entity in the geometric model 
and the field of view is its extent. First, the respective geometric relationships 
between the entity frame representing the camera and the frames of all the other 
entities in the model are computed in terms of the camera-entity's coordinate 
frame. The extents defining the other entities are then transformed into the 

20 camera-entity's coordinate frame. A standard polygon intersection technique (or 
similar procedure if the extents are characterized by a planar shape other than a 
polygon) would then be employed to determine if any portion of the other 
entities' extents overlap the extent of the camera-entity representing its field of 
view. Those entities having extents that are discovered to overlap would be 

25 reported as being within the camera's field of view to the program making the 
query. 



As a further example, consider a query that requests if a person can 
clearly see a display screen. One way to handle this query would be to define a 
3 0 triangular region in terms of the person-entity's coordinate frame that has one 
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vertex corresponding to a point midway between the person's eyes and its other 
two vertices corresponding to points at opposite sides of the display screen. This 
could be done by first querying for the geometric relationship between the 
person and the display, and utilizing information about the size of the display to 
5 determine where the edges should be located. The process would proceed as 
described above by computing the relationships between the frames of each 
entity in the environment and the frame of the person-entity. The extents which 
define the other entities' are transformed using these computed relationships 
into the person-entity's coordinate frame. A polygon intersection technique (or 

1 o similar procedure if the defined region and extents are characterized by a planar 

shape other than a polygon) would then be used to determine if any portion of 
the other entities' extents overlap the newly defined region. The entities which 
intersect the original triangular region would be reported back to the querying 
program Since extents in the model may represent non-physical things such as 
15 regions of service (which would not obscure view of a screen), it could be left up 
to the querying program to determine which of the intersecting extents actually 
represent things which obscure vision. For example, an entity which represents 
a field of view of a camera would not obscure a person's sight of a display. 

2 0 The foregoing process for handling queries concerning a request to find 

an entity or set of entities whose extents have a particular geometric relationship 
to a prescribed region or extent associated with a given entity is summarized in 
the flow diagram of Fig. 9. The first action 900 of the process involves waiting 
for queries concerning the aforementioned request. Periodically, it is 

25 determined whether such a query was received as indicated in process action 
902. If no query was received, the waiting and checking actions 900 and 902 
are continued. If, however, such a query was received, then the relative 
geometric relationship is computed between the frame of the given entity and the 
frames of all other entities connected to it by a chain of measurements (process 

30 action 904). Next, the coordinates of the extents associated with these other 
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entities are transformed into the coordinate frame of the given entity using the 
respective relative geometric relationships computed previously between the 
given entity and the other entities (process action 906). A region intersection 
process is then employed to determine if the extents associated with the other 
5 entities involved in the query intersect the prescribed region or extent associated 
with the given entity (process action 908). Finally, in process action 910 the 
intersection information (i.e. which entities along with optionally the region of 
intersection of each extent) is provided to the requesting source. The monitoring 
is continued and foregoing procedure repeated every time a new query of the 
10 aforementioned type is received. 



Another interesting aspect of the last example is that even if there are no 
overlapping extents, this does not mean that the person is actually facing toward 
the display screen. The purpose for the query may be that the querying program 

15 wants to display something intended for the person to see. This is where the 

aforementioned rotational coordinates used to indicate the "heading" of an entity 
could come into play. For instance, if the rotation component associated with 
the person indicates that the display screen would be in his or her field of view, 
then this would be reported. If not, then it could be reported to the querying 

2 0 program that the person could not see the display screen. The same procedure 
could be performed in the context of the display screen to ensure that it is 
oriented such that the person can see it. 



It is noted that a query could be of a one-time or "one-shot" nature, or 
25 may be a standing query. For a one-time query, the querying program would 
request the desired geometric information and receive a one-time response. 
However, a standing query would be one predicated on some event, and every 
time this event occurred, a response to the query would be provided. For 
example, the previously discussed query concerning whether a person could 
30 clearly see a display screen might be requested to be responded to every time 
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the person of interest is added to the geometric model of the environment (e.g., 
enters a room), it is also noted that because the environment is dynamic the 
response to such a query may have to be re-computed each time it is responded 
to, rather than relying on the previous response. 

5 
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Wherefore, what is claimed is: 

1 . A computer-implemented process for providing a geometric model 
5 database for use in a ubiquitous computing environment to respond to queries 

about the environment's geometric state, comprising using a computer to 
perform the following process actions: 

accepting information about the geometric state of the 

environment; 

1 o building a geometric model database of the environment based on 

an initial input of said information; 

maintaining the geometric model database by modifying it based 
on the input of updated information about the geometric state of the 
environment; and 

15 responding to queries concerning the geometric relationships 

between entities in the environment using the geometric model database. 

2. The process of Claim 1 , wherein the process action of accepting 
information about the geometric state of the environment, comprises the actions 

2 0 of: 

inputting identifying information from an external source concerning 
an object existing in the environment, referred to as an entity, which is to be 
included in the geometric model database, said information comprising the 
entity's extent which is one of (i) the physical size of the entity, or (ii) the service 
2 5 region of the entity; and 

inputting measurements, each of which defines the entity's 
relationship to one other entity in the geometric model database. 
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3. The process of Claim 2, wherein the entity represents a camera 
and the camera's extent corresponds to a service region constituting a field of 
view of the camera. 

5 4. The process of Claim 1 , wherein the process action of building the 

geometric model database, comprises the actions of: 

establishing a set of entities that are of interest in the environment; 

representing each entity by a coordinate frame and an extent, 
wherein said extent defines one of (i) the physical size of the entity, or (ii) the 

1 o service region of the entity; and 

characterizing the location of each entity in the environment 
relative to other entities using a measurement defining the entity's relationship to 
at least one of said other entities. 

15 5. The process of Claim 4, wherein the process action of establishing 

a set of entities, comprises the actions of: 

accepting identifying information from an external source 
concerning an object existing in the environment, referred to as an entity, which 
is to be included in the geometric model database, said information comprising 
20 the entity's extent assigning a unique entity identifier to each entity which is 
then used by the geometric model database and the external source in referring 
to the entity; and 

making the entity identifiers available to the external source. 

2 5 6. The process of Claim 5, wherein an external source provides more 

than one extent for an entity, and wherein the process action of assigning a 
unique entity identifier to each entity, comprises the actions of: 

assigning a separate identifier to each entity-extent combination; 

and 
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setting the measurement between entity-extent combinations 
associated with the same entity to zero. 

7. The process of Claim 4, wherein the process action of representing 
each entity by a coordinate frame and an extent, comprises a process action of 
representing each entity by a coordinate frame having a fixed geometric 
relationship to the physical object associated with the entity. 

8. The process of Claim 4, wherein the process action of representing 
each entity by a coordinate frame and an extent, comprises a process action of 
characterizing an entity's extent as a polygonal region within the environment 
defined in terms of the entity's coordinate frame whenever the external source 
provides information as to the shape of the entity's extent. 

9. The process of Claim 4, wherein the process action of representing 
each entity by a coordinate frame and an extent, comprises a process action of 
characterizing an entity's extent as a line segment within the environment 
defined in terms of the entity's coordinate frame whenever the external source 
provides information indicating the entity's extent to be such a line segment. 

10. The process of Claim 4, wherein the process action of representing 
each entity by a coordinate frame and an extent, comprises a process action of 
characterizing an entity's extent as a point coincident with the origin of the 
entity's coordinate frame whenever the external source fails to provide 
information defining a size for the entity's extent. 

1 1 . The process of Claim 4, wherein the process action of representing 
each entity by a coordinate frame and an extent, comprises a process action of 
characterizing an entity's extent as a point having a prescribed geometric 
relationship to the origin of the entity's coordinate frame. 
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12. The process of Claim 4, wherein the process action of 
characterizing the location of each entity in the environment relative to other 
entities using a measurement defining the entity's relationship to one of said 
other entities, comprises an action of using a measurement specifying the 
position and orientation of each other entity's coordinate frame origin in terms of 
the coordinate frame of the entity under consideration. 

13. The process of Claim 4, wherein the process action of 
characterizing the location of each entity in the environment relative to other 
entities using a measurement defining the entity's relationship to at least one of 
said other entities, comprises the actions of: 

assigning a unique measurement identifier to each measurement 
which is then used by the geometric model database and the external source in 
referring to the measurement defining the entity's relationship to another entity; 
and 

making the measurement identifiers available to the external 

source. 

14. The process of Claim 12, wherein the process action of using a 
measurement specifying the position and orientation of each other entity's 
coordinate frame origin in terms of the coordinate frame of the entity under 
consideration, comprises an action of assigning a spatial uncertainty estimate to 
the measurement which is indicative of the accuracy of the method used to 
obtain the measurement. 

15. The process of Claim 14, wherein each measurement is provided 
to the geometric model database by an external source, and wherein more than 
one measurement defining an entity's relationship to another entity may be 
provided by separate external sources, and wherein the process action of 



characterizing the location of each entity in the environment relative to other 
entities using a measurement, comprises an action of, whenever more than one 
measurement defining an entity's relationship to another entity is received, using 
only the measurement having the lower uncertainty. 

5 

16. The process of Claim 14, wherein each measurement is provided 
to the geometric model database by an external source, and wherein more than 
one measurement defining an entity's relationship to another entity may be 
provided by separate external sources, and wherein the process action of 

10 characterizing the location of each entity in the environment relative to other 

entities using a measurement, comprises an action of, whenever more than one 
measurement defining an entity's relationship to another entity is received, 
arbitrarily choosing one of the measurements for use in characterizing the 
locations. 

15 

17. The process of Claim 14, wherein each measurement is provided 
to the geometric model database by an external source, and wherein more than 
one measurement defining an entity's relationship to another entity may be 
provided by separate external sources, and wherein the process action of 

20 characterizing the location of each entity in the environment relative to other 

entities using a measurement, comprises an action of, whenever more than one 
measurement defining an entity's relationship to another entity is received, 
combining said measurements using their relative uncertainties as weights. 

2 5 18. The process of Claim 4, wherein the process action of 

characterizing the location of each entity in the environment relative to other 
entities using a measurement defining the entity's relationship to one of said 
other entities, comprises an action of revising the measurements, said revising 
action comprising the actions of: 
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inputting a spatial uncertainty estimate associated with each 
measurement which is indicative of the accuracy of the method used to obtain 
the measurement; 

identifying cycles of measurements among the measurements, 
5 wherein a cycle of measurements is defined as a string of measurements 
starting at the origin of a first entity frame in the cycle and following 
measurements from one entity to the next until reaching a last measurement in 
the cycle representing the relationship between the coordinate frame of a next to 
last entity of the cycle and the origin of said first entity frame; and 
10 for each cycle identified, computing the difference between a given 

location of said first entity frame's origin and the location of that origin as 
indicated by following the chain of measurements making up the cycle; 

computing revised measurements for the identified cycles by 
simultaneously adjusting the measurements based on their associated 
15 uncertainty estimates so as to make the difference in the given location of said 
first entity frame's origin in each of the identified cycles match the location of that 
origin as indicated by following the chain of measurements making up the cycle 

19. The process of Claim 18, further comprising a process action of 
2 0 repeating the process actions of inputting spatial uncertainty estimates, 

identifying cycles of measurements, computing the difference between a given 
location of said first entity frame's origin and the location of that origin as 
indicated by following the chain of measurements making up the cycle for each 
cycle identified, and computing revised measurements, whenever new 
25 measurements are provided. 



20. The process of Claim 18, further comprising a process action of 
repeating the process actions of inputting spatial uncertainty estimates, 
identifying cycles of measurements, computing the difference between a given 
3 0 location of said first entity frame's origin and the location of that origin as 
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indicated by following the chain of measurements making up the cycle for each 
cycle identified, and computing revised measurements, on a periodic basis. 

21 . The process of Claim 4, wherein the process action of 
5 characterizing the location of each entity in the environment relative to other 
entities using a measurement defining the entity's relationship to at least one of 
said other entities, comprises an action of detecting errors in the measurements, 
said error detecting action comprising the actions of: 

(a) inputting a spatial uncertainty estimate associated with the 
1 0 measurement which is indicative of the accuracy of the method used to obtain 

the measurement; 

(b) identifying cycles of measurements among the 
measurements, wherein a cycle of measurements is defined as a string of 
measurements starting at the origin of a first entity frame in the cycle and 

15 following measurements from one entity to the next until reaching a last 

measurement in the cycle representing the relationship between the coordinate 
frame of a next to last entity of the cycle and the origin of said first entity frame; 
and 

(c) for each cycle identified, 

2 0 computing the location of said first entity frame's origin as 

indicated by following the chain of measurements making up the cycle, along 
with computing an uncertainty region around the computed location of the origin 
based on a combination of the uncertainty estimates associated with each 
measurement in the cycle, 

25 determining if a given location of said first entity frame's 

origin is within the computed uncertainty region; 

whenever the given location falls outside the uncertainty 
region, declaring that at least one of the measurements in the cycle is incorrect, 
and 
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whenever it is declared that one of the measurements in the 
cycle is incorrect, disregarding these measurements and requesting that 
replacement measurements be provided. 

5 22. The process of Claim 21 , further comprising a process action of 

repeating process action (c) whenever replacement measurements are provided. 

23. The process of Claim 21 , further comprising a process action of 
repeating process action (c) periodically. 

10 

24. The process of Claim 4, wherein the process action of 
characterizing the location of each entity in the environment relative to other 
entities using a measurement defining the entity's relationship to one of said 
other entities, comprises an action of revising the measurements, said revising 

15 action comprising the actions of: 

(a) inputting a spatial uncertainty estimate associated with each 
measurement which is indicative of the accuracy of the method used to obtain 
the measurement; 

(b) identifying cycles of measurements among the 

2 0 measurements, wherein a cycle of measurements is defined as a string of 
measurements starting at the origin of a first entity frame in the cycle and 
following measurements from one entity to the next until reaching a last 
measurement in the cycle representing the relationship between the coordinate 
frame of a next to last entity of the cycle and the origin of said first entity frame; 

25 (c) for each cycle identified, 

computing the location of said first entity frame's 
origin as indicated by following the chain of measurements making up the cycle, 
along with computing an uncertainty region around the computed location of the 
origin based on a combination of the uncertainty estimates associated with each 

3 0 measurement in the cycle, 
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determining if a given location of said first entity 
frame's origin is within the computed uncertainty region, 

whenever the given location falls outside the 
uncertainty region, declaring that at least one of the measurements in the cycle 
5 is incorrect, and 

whenever it is declared that one of the measurements 
in the cycle is incorrect, disregarding these measurements and requesting that 
replacement measurements be provided, and 

(d) computing revised measurements for those identified cycles 

10 in which the given location of the origin is not the same as its computed location 
but in which the given location falls within the uncertainty region by 
simultaneously adjusting the measurements based on their associated 
uncertainty estimates so as to make the difference in the given location of said 
first entity frame's origin in each of the identified cycles match the location of that 

15 origin as indicated by following the chain of measurements making up the cycle. 

25. The process of Claim 24, further comprising a process action of 
repeating process actions (a) through (d) whenever new measurements are 
provided. 

20 

26. The process of Claim 24, further comprising a process action of 
repeating process actions (a) through (d) periodically. 

27. The process of Claim 4, wherein the process action of responding 
25 to queries concerning the geometric relationships between entities in the 

environment, comprises an action of, upon receiving a request from an external 
source to identify the extent of a particular entity, providing the extent 
information to the external source. 
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28. The process of Claim 4, wherein the process action of responding 
to queries concerning the geometric relationships between entities in the 
environment, comprises the actions of: 

waiting for incoming queries from external sources for requests 
5 concerning the relative geometric relationship between two entities; 

whenever a request concerning the relative geometric relationship 
between two entities is received, determining if a direct measurement exists 
between the two entities involved in the request; 

whenever said direct measurement exists, providing information 
1 0 concerning the measurement to the external source making the request. 

29. The process of Claim 4, wherein the process action of responding 
to queries concerning the geometric relationships between entities in the 
environment, comprises the actions of: 

15 waiting for incoming queries from external sources for requests 

concerning the relative geometric relationship between two entities; 

whenever a request concerning the relative geometric relationship 
between two entities is received, determining if a direct measurement exists 
between the two entities involved in the request; 

20 whenever said direct measurement does not exist, employing a 

breadth-first search to find a measurement path between the two entities 
involved in the request that has the fewest number of measurement links, 
wherein a measurement path is a chain of measurements from a first of the two 
entities involved in the request, through at least one intermediate entity, to the 

25 other entity involved in the request; 

computing the requested measurement information using the 
measurements in said measurement path, if one was found; and 

providing the computed measurement information to the external 
source making the request. 
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30. The process of Claim 29, wherein the process action of responding 
to queries concerning the geometric relationships between entities in the 
environment, further comprises, whenever as a result of employing the breadth- 
first search more than one measurement path is discovered having the same 

5 fewest number of measurement links, randomly choosing one of the discovered 
measurement path for use in computing the requested measurement information. 

31 . The process of Claim 29, wherein the process action of using a 
measurement specifying the position and orientation of each other entity's 

1 0 coordinate frame origin in terms of the coordinate frame of the entity under 

consideration, comprises an action of assigning a spatial uncertainty estimate to 
the measurement which is indicative of the accuracy of the method used to 
obtain the measurement, and wherein the process action of responding to 
queries concerning the geometric relationships between entities in the 

15 environment, further comprises, whenever as a result of employing the breadth- 
first search more than one measurement path is discovered having the same 
fewest number of measurement links, choosing the discovered measurement 
path exhibiting a lowest combined uncertainty for use in computing the 
requested measurement information. 

20 

32. The process of Claim 4, wherein the process action of responding 
to queries concerning the geometric relationships between entities in the 
environment, comprises the actions of: 

inputting incoming queries from external sources for requests to 

2 5 find entities whose extents have a particular geometric relationship to a 

prescribed region or extent associated with a specified entity 

ascertaining the relative geometric relationship between the frame 

origin of the specified entity and the frame origins of other entities 

transforming the coordinates of the extents associated with the 

3 0 other entities into the coordinate frame of the specified entity using the 
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respective relative geometric relationships between the frame origin of the 
specified entity and the frame origins of said other entities; 

employing a region intersection procedure to determine if the 
extents associated with said other entities intersect the prescribed region or 
5 extent associated with the specified entity; and 

providing information to the requesting source as to whether the 
extents of any of said other entities intersect the prescribed region or extent of 
the specified entity, and if so which of the other entities' extents intersect. 



1 0 33. The process of Claim 4, wherein the prescribed region or extent 

associated with the specified entity, as well as the extents associated with said 
other entities, are characterized by polygonal areas, or degenerated version 
thereof constituting a line segment or point, and wherein the region intersection 
procedure is a polygon intersection procedure. 
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34. The process of Claim 1 , wherein the process action of responding 
to queries concerning the geometric relationships between entities in the 
environment, comprises an action of, upon receiving a standing request from an 
external source, responding to the request each time a prescribed event occurs. 



35. A system for providing a geometric model database for use in a 
ubiquitous computing environment to respond to queries about the 
environment's geometric state, comprising: 

at least one general purpose computing device; and 
25 a computer program comprising program modules executable by 

the computing device or devices, wherein the computing device or devices are 
directed by the program modules of the computer program to, 

input information about the geometric state of the 
environment from at least one external source, 
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establish a set of entities that represent objects in the 
environment based on an initial input of said information, 

represent each entity by a coordinate frame and an extent, 
wherein said extent is based on an initial input of said information, 

characterize the location of each entity in the environment 
relative to other entities using a measurement defining the entity's relationship to 
at least one of said other entities. 

36. The system of Claim 35, further comprising: 

a program module for storing as initializing data in a non-volatile 
initializing database, information concerning the entities, their extents, and the 
measurements between entities contained within the geometric model database; 
and wherein, 

the program module for inputting information about the geometric 
state of the environment, comprises an action of inputting the stored initializing 
data from the non-volatile database at the start of said process for providing a 
geometric model database. 

37. The system of Claim 36, wherein the program module for storing 
initializing data, comprises a sub-module for storing only information concerning 
entities, extents, and measurements that is anticipated not to change 
substantially over time. 

38. The system of Claim 37, wherein the program module for inputting 
information about the geometric state of the environment, comprises a sub- 
module for inputting update information characterizing a current geometric state 
of the environment. 

39. The system of Claim 38, wherein the program module for storing 
initializing data, comprises a sub-module for storing information concerning the 
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entities, their extents, and the measurements representative of the most current 
geometric state of the environment. 

40. The system of Claim 36, wherein the program module for 
5 establishing a set of entities comprises a sub-module for assigning a unique 
entity identifier to each entity entered into the geometric model database, which 
is then used by the geometric model database and external sources in referring 
to the entity, and wherein the program module for characterizing the location of 
each entity in the environment relative to other entities using a measurement 
10 comprises a sub-module for assigning a unique measurement identifier to each 
measurement entered into the geometric model database, which is then used by 
the geometric model database and the external sources in referring to the 
measurement. 

15 41 . The system of Claim 40, wherein the program module for storing 

initializing data, further comprises sub-modules for: 

storing the entity and measurement identifiers assigned to the 

entities and measurements comprising the initializing data in a non-volatile 

initializing database; and 
2 0 making the entity and measurement identifiers available to the 

external sources. 

42. A computer-readable medium having computer-executable 
instructions for providing a geometric model database for use in a ubiquitous 
2 5 computing environment to respond to queries about the environment's geometric 
state, said computer-executable instructions comprising: 

inputting information about the geometric state of the 
environment from at least one external source, 

building a geometric model database of the environment 
30 based on an initial input of said information, and 
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maintaining the geometric model database by modifying it 
based on the input of updated information about the geometric state of the 
environment. 



5 43. The computer-readable medium of Claim 42, wherein the 

instruction for building the geometric model database, comprises sub-modules 
for: 

establishing a set of entities representing objects in the 

environment; 

10 representing each entity by a coordinate frame and an extent; and 

characterizing the location of each entity in the environment 
relative to other entities using a measurement defining the entity's relationship to 
one of said other entities. 



15 44. The computer-readable medium of Claim 43, wherein the 

instruction for inputting information about the geometric state of the environment, 
comprises a sub-module for inputting update information characterizing a current 
geometric state of the environment. 



2 0 45. The computer-readable medium of Claim 44, wherein the 

instruction for maintaining the geometric model database, comprises a sub- 
module for updating the geometric model database on an on-going basis, using 
the inputted update information characterizing a current geometric state of the 
environment, to ensure to the best degree possible given the update information, 

25 that only entities currently existing in the environment and their associated 

current extents are included in the database, and that measurements between 
the current entities are representative of the current geometric relationships 
between the current entities. 
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46. The computer-readable medium of Claim 45, wherein the sub- 
module for establishing a set of entities, comprises sub-modules for: 

assigning a unique entity identifier to each entity entered into the 
geometric model database, which is then used by the geometric model database 
5 and external sources in referring to the entity; and 

making the entity identifiers available to the external sources. 

47. The computer-readable medium of Claim 46, wherein the sub- 
module for establishing a set of entities, further comprises a sub-module for 

10 deactivating existing entity identifiers associated with entities that are indicated 
in said update information as no longer being in the environment. 



48. The computer-readable medium of Claim 47, wherein the sub- 
module for characterizing the location of each entity in the environment relative 
15 to other entities using a measurement, comprises sub-modules for: 

assigning a unique measurement identifier to each measurement 
entered into the geometric model database, which is then used by the geometric 
model database and external sources in referring to the measurement; and 
making the measurement identifiers available to the external 

2 0 sources. 



49. The computer-readable medium of Claim 48, wherein the sub- 
module for deactivating existing entity identifiers associated with entities that are 
indicated in said update information as no longer being in the environment, 
2 5 comprises sub-modules for: 

deleting all measurements associated with a deactivated entity 
from the geometric model database; and 

retiring the measurement identifier assigned to any measurement 
that is deleted for being associated with a deactivated entity. 
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50. The computer-readable medium of Claim 43, wherein the sub- 
module for characterizing the location of each entity in the environment relative 
to other entities using a measurement, comprises sub-modules for: 

assigning a unique measurement identifier to each measurement 
5 entered into the geometric model database, which is then used by the geometric 
model database and external sources in referring to the measurement; and 
making the measurement identifiers available to the external 

sources. 



10 51 . The computer-readable medium of Claim 50, wherein the sub- 

module for characterizing the location of each entity in the environment relative 
to other entities using a measurement, further comprises sub-modules for: 

whenever a new current measurement is provided in said inputted 
update information, using it to replace the corresponding measurement already 
15 existing in the geometric model database; and 

assigning the measurement identifier associated with the existing 
measurement to the new current measurement. 
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GEOMETRIC MODEL DATABASE FOR USE IN UBIQUITOUS COMPUTING 

ABSTRACT OF THE DISCLOSURE 

5 A system and process for providing a geometric model database for use 

in an ubiquitous computing environment. In general, the geometric model 
database system and process is capable of accepting information about the 
geometric state of the environment, building a geometric model of this 
environment, maintaining and storing the geometric model, and handling queries 

1 0 about the environment's geometric state. The task of building a geometric model 
begins by establishing a set of entities that are of interest in the environment. 
An entity represents an object which exists in the physical world. In the 
geometric model database, an entity is represented by a coordinate frame and 
an extent. Extents refer to the physical size, or some service region such as a 

15 field of view, associated with an entity. The location of an entity in the physical 
world is defined using "measurements". In general, a measurement is simply a 
mathematical description of the geometric relationship between two entities. 
More precisely, a measurement describes the position and orientation of one 
entity's coordinate frame, expressed in terms of another entity's coordinate 

2 0 frame. Measurements originating at an entity's frame are expressed in terms of 
that frame. While various mathematical representations of the geometric 
relationship between entities could be employed, a preferred one characterizes 
a measurement as the relative position, and the relative orientation or heading, 
of two entities along with a covariance matrix which describes the uncertainty in 

25 these values. 
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Selected Cycle Around The Cycle From 
The Chosen First Frame, While Using 
The Uncertainty Estimates Associated 
With Each Measurement To Define An 
Uncertainty Region Around The 
Computed Location 
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Does The ^<^708 
"Specified Location Of The 
First Entity Frame Fall Within The 
Computed Uncertainty 
Region 
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Deem That At Least One Of The 
Measurements Making Up The 
Selected Cycle Is Incorrect 
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Disregard All The Measurements That 
Are Part Of The Selected Cycle 
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Request That New Measurements Be 
Provided 
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Deem The 
Measurements Of The 
Selected Cycle To Be 

Accurate 
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Wait For Queries Concerning The 
Relative Geometric Relationship 
Between Two Entities 
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FIG. 8 
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Provide The 
Requested 
Measurement To 
The Source Of The 
Query 
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Employ A Breadth-First Search To Find 
The Measurement Path Between The 
Two Entities That Has The Fewest 
Number Of Measurement Links 
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Inform The Query 
Source That The 

Requested 
Information Cannot 
Be Provided 
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Compute The Requested Measurement 
' Using The Measurements In The 
Discovered Path 
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Provide The Requested Measurement 
To The Query Source 
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Wait For Queries Concerning A 
Request To Find An Entity Or Entities 
Whose Extents Have A Particular 
Geometric Relationship To A 
Prescribed Region Or Extent 
Associated With A Given Entity 
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Compute The Relative Geometric 
Relationship Between The Frame Of 
The Given Entity And The Frames Of 
All Other Entities Connected To It By A 
Chain Of Measurements 
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Transform The Coordinates Of The 
Extents Associated With The Other 
Entities Into The Coordinate Frame Of 
The Given Entity 
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Employ A Region Intersection Process 
To Determine If The Extents 
Associated With The Other Entities 
Intersect The Prescribed Region Or 
Extent Of The Given Entity 
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Provide The Intersection Information 
To The Requesting Source 
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A. 

PATENT 

Microsoft Docket No. 146837.2 
LH&DNo. MCS-1 20-99 



DECLARATION AND POWER OF ATTORNEY FOR PATENT APPLICATION 



As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name. I believe I am 
the original, first and sole inventor (if only one name is listed below) or an original, first and joint inventor 
(if plural names are listed below) of the subject matter which is claimed and for which a patent is sought 
on the invention entitled: 

GEOMETRIC MODEL DATABASE FOR USE IN UBIQUITOUS COMPUTING 



MThis declaration is of the following type: 

% [X] original 

■vj [] divisional 

a [] continuation 

Sj [] continuation-in-part 

&\he specification of which 

□(check one) is attached hereto 

4} was filed on 

y Application Serial No. 

J? and was amended on 

% (if applicable) 

1 hereby state that I have reviewed and understand the contents of the above-identified specification, 
including the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose information which is material to the examination of this application in 
accordance with Title 37, Code of Federal Regulations §1. 56(a). 

I do not know and do not believe that the invention was ever known or used in 
the United States of America before my or our invention thereof; 

I do not know and do not believe that the invention was ever patented or 

described in any printed publication in any country before my or our invention thereof or more 
than one year prior to this application; 

I do not know and do not believe that the invention was in public use or on 

sale in the United States of America more than one year prior to this application. 
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The invention has not been patented or made the subject of any inventor's 

certificate issued before the date of this application in any country foreign to the United States of 
America on an application filed by me or my legal representatives or assigns more than twelve 
months prior to this application; and 

As to applications for patents or inventor's certificate on the invention 

filed in any country foreign to the United States of America, prior to this application by me or my 
legal representatives or assign: 
XX no such applications have been filed, or 
such applications have been filed as follows: 



PRIORITY CLAIM (35 U.S.C. §119) 

We hereby claim foreign priority benefits under Title 35, United States Code, §119, of any foreign 
application(s) for patent or inventor's certificate or of any PCT international application(s) designating at 
least one country other than the United States of America listed below, and have also identified below 
any foreign application(s) for patent or inventor's certificate or any PCT international application(s) 
designating at least one country other than the United States of America filed by me on the same 
subject matter having a filing date before that of the application(s) of which priority is claimed. 

A. Prior foreign/PCT application(s) filed within 12 mos. (6 mos. for design) prior to this 
^application, and any priority claims under 35 U.S.C. §119 

Cpountry/PCT Application No. Date Filed Priority Claimed 

rNONE rjYes [] No 

o []Yes []No 

O fj Yes []No 

JB. All foreign application(s), if any, filed more than 12 mos. (6 mos. for design) prior to this 
JU.S. application 

> Country: NONE 
Application No.: 
Filing Date: 

PRIOR U.S. APPLICATION(S) FOR WHICH BENEFIT 
UNDER 35 U.S.C. §11 9(e) or 35 U.S.C. §120 IS CLAIMED 

Serial No. Filing Date Patented Pending Abandoned Provisional Application 

60/170,285 12/10/99 X 



POWER OF ATTORNEY 

I hereby appoint the following attorneys and/or agents to prosecute this application and transact all 
business in the Patent and Trademark Office connected therewith: 
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Katie E. Sako 
Daniel D. Crouse 
Richard T. Lyon 
Edmond A. DeFrank 

Send correspondence to: 

LYON, HARR & DEFRANK 
300 Esplanade Drive, Suite 800 
Oxnard, CA 93030 
FAX: (805) 271-0255 



Registration No. 32,628 

Registration No. 32,022 
Registration No. 37,385 

Registration No. 37,814 

Direct telephone calls to: 

Richard T. Lyon 

(805) 278-8855 



DECLARATION 

I/We hereby declare that all statements made herein of our own knowledge are true and that all 
statements made on information and belief are believed to be true; and, further, that these 
statements were made with the knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under Sec. 1001 of Title 18 of the United States 
Code, and that such willful false statements may jeopardize the validity of the application or any 
ypatents issued thereon. 



m Full name of sole or first invenJor 

Inventor's signature: 
M Date: 

* Residence: 



^ Citizenship: 

G Post Office Address: 




Barrett L B&umitt 




4010 266 1 " Avenue NE 



Redmond, Washington 98053 



USA 



4010 266 th Avenue NE, Redmond, Washington 98053 



Full name of second inventor: 

Inventor's signature: 

Date: 

Residence: 



Citizenship: 

Post Office Address: 



Steven A. N. Shafer 



1314 Fifth Ave. West 



Seattle, Washington 98119 



USA 



1314 Fifth Ave. West, Seattle, Washington 981 1 9 



Full name of third inventor: Brian R. Meyers 

Inventor's signature: Jb^.n 

Date: 

Residence: 4040 W. Lk Sam Pky SE 

Bellevue, Washington 9800$f ff 

Citizenship: USA 

Post Office Address: 4040 W. Lk Sam Pky SE, Bellevue, Washington 9800ff g 

(Declaration ends with this paqp) 
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